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Abstract 


This  report  presents  the  mechanics  of  performing  a  verification  and 
validation  analysis  for  practical  problems,  focusing  mainly  on  the  estimation  of 
uncertainty  in  the  numerical  prediction  due  to  the  use  of  finite  grid  sizes.  A 
detailed  example  is  presented  along  with  discussions  regarding  many  of  the 
practical  issues  involved  in  performing  a  rigorous  verification  and  validation 
analysis.  The  particular  approach  outlined  in  this  report  is  mainly  based  on 
theoretical  work  performed  at  the  Iowa  Institute  of  Hydraulic  Research.  This 
approach  requires  numerical  solutions  on  3  related  grids;  however,  an 
alternative  approach  requiring  solutions  on  only  2  grids  is  also  demonstrated. 

Administrative  Information 

The  work  described  in  this  report  was  performed  by  the  Propulsion  and  Fluid  Systems 
Department  (Code  54)  of  the  Hydromechanics  Directorate  at  the  Naval  Surface  Warfare  Center, 
Carderock  Division  (NSWCCD).  The  work  was  funded  by  the  Office  of  Naval  Research,  Code 
333  as  part  of  the  ONR  Surface  Ship  Accelerated  Hydro  S&T  Project  under  program  element 
61153N. 


Introduction 

Computational  fluid  dynamics  (CFD)  solutions  are  increasingly  being  used  as  input  for  the 
engineering  design  process  within  the  naval  hydrodynamics  community.  Naval  combatants  in 
particular  are  going  through  revolutionary  design  changes  for,  among  other  things,  improved 
stealth  [Valenti,  2001].  The  new  combatant  concepts  are  sufficiently  different  from  traditional 
hull  forms  that  existing  databases  and  empirical  formulas  typically  used  for  design  are  no  longer 
applicable.  This  has  required  the  evolution  of  a  new  paradigm  for  ship  design  that  is  more  reliant 
than  ever  on  CFD  [Rood,  2000].  In  the  past  Reynolds-averaged  Navier-Stokes  (RANS) 
numerical  predictions  have  mainly  been  used  to  evaluate  trends  and  differences  between  related 
geometries.  However,  the  drive  for  more  reliance  on  numerical  predictions  is  creating  a  desire  to 
obtain  absolute  values  from  these  computations  as  well  as  a  better  understanding  of  their 
accuracy.  This  is  motivating  the  development  of  methods  for  performing  verification  and 
validation  (V&V)  analyses  [Roache,  1998]  to  assess  the  numerical  and  modeling  uncertainty  of 
these  predictions. 

The  approach  presented  in  this  report  is  based  on  the  work  of  Stem  et  al.  [1999].  The 
interested  reader  should  consult  Stem  et  al.  [1999]  for  details  of  the  theory  and  meaning  of  the 
verification  and  validation  process.  This  particular  approach  is  just  one  of  various  approaches 
being  put  forth  in  the  literature.  Indeed,  much  interest  has  been  paid  to  the  topic  of  verification 
and  validation  in  recent  years.  Verification  and  validation  policies  and  guidelines  have  been 
developed  by  various  organizations  including  ALA  A  [1998]  and  ASME  [Freitas,  1993]  along 
with  numerous  journal  papers  dedicated  to  the  topic.  Concurrently,  there  are  efforts  by  the 
experimental  community  to  provide  benchmark  data  sets  for  V&V.  Recent  measurements  taken 
for  DTMB  Model  5415  are  an  example  of  this  effort  [Ratcliffe,  2000].  Although  there  is  no 
universally  accepted  V&V  procedure,  the  V&V  process  is  the  beginning  of  a  methodology  to 
provide  some  information  on  the  level  of  confidence  in  a  computational  solution  and  will  most 
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likely  become  another  important  factor  to  be  used  by  the  community  in  evaluating  a 
computational  solution. 


Procedure 


The  verification  and  validation  procedure  presented  here  involves  making  uncertainty 
estimates  of  numerical  predictions  at  the  location  of  experimental  data  points.  Using  the 
terminology  of  Stem  et  al.  [1999],  verification  is  defined  as  the  process  of  assessing  the 
numerical  uncertainty  (Usn)-  Typically,  the  most  significant  sources  of  numerical  uncertainty  in 
a  RANS  calculation  come  from  the  use  of  finite  grid  sizes,  finite  time  steps,  and  the  use  of 
iterative  solution  methods.  The  majority  of  this  report  focuses  on  the  estimation  of  the 
uncertainty  in  the  numerical  prediction  due  to  the  influence  of  finite  grid  sizes.  This  uncertainty 
estimate  requires  that  numerical  solutions  be  obtained  on  more  than  one  grid.  Validation  is 
defined  as  the  process  of  assessing  simulation  modeling  uncertainty.  Simulation  modeling 
uncertainty  is  caused  by  assumptions  and  approximations  in  the  mathematical  representation  of 
the  physical  problem.  As  an  example,  one  common  source  of  simulation  modeling  uncertainty 
would  be  the  use  of  a  turbulence  model.  In  this  procedure  assessing  the  simulation  modeling 
uncertainty  is  accomplished  by  comparison  with  benchmark  experimental  data.  Stem  et  al. 
[1999]  defines  a  quantity  called  the  validation  uncertainty  (Uv). 
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The  terms  on  the  right  hand  side  of  this  equation  are  defined  as  follows. 


Uo=Estimated  experimental  data  uncertainty 
UspD=Estimated  simulation  previous  data  uncertainty 
UsN=Estimated  simulation  numerical  uncertainty 


In  this  procedure  these  uncertainty  estimates  are  made  at  the  location  of  the  experimental  data 
points  and  the  validation  procedure  is  accomplished  by  determining  if  the  difference  between  the 
numerical  prediction  and  benchmark  experimental  data,  known  as  the  comparison  error,  is  within 
the  validation  uncertainty  at  these  locations.  Thus,  a  numerical  prediction  is  considered  validated 
“at  the  Uv  level”  if, 

|£|  <uv 

where, 

E  =  D-S  =  Comparison  Error 
D  =  Experimental  data 

S  =  Simulation  result  (i.e.  numerical  prediction) 


This  report  assumes  that  the  estimate  of  the  uncertainty  in  the  experimental  data  (Up)  has 
been  provided  by  the  experimentalists  and  that  the  estimate  of  simulation  previous  data 
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uncertainty  (USpd)  is  known  or  is  negligible.  Simulation  previous  data  uncertainty  could  come 
from  such  things  as  uncertainty  in  fluid  properties  or  uncertainty  in  any  experimental  data  used 
as  input  for  the  numerical  simulation.  This  report  will  therefore  focus  on  the  simulation 
numerical  uncertainty  (USn)-  USn  itself  is  a  combination  of  other  uncertainties  of  which  the  two 
most  significant  for  steady  state  solutions,  grid  uncertainty  (Ug)  and  iterative  convergence 
uncertainty  (Ui),  are  considered  in  this  report. 

U2SN=UG+U1 


where, 

Uo=Estimation  of  grid  uncertainty 
UpEstimation  of  iterative  convergence  uncertainty 


It  is  recommended  that  whenever  possible  the  numerical  solution  be  run  to  a  point  where 
iterative  convergence  errors  become  negligible.  It  is  recognized,  however,  that  for  complex 
problems  this  may  not  be  possible  for  all  variables,  in  all  areas,  of  all  grids.  When  iterative 
convergence  uncertainty  is  on  the  finest  grid  it  can  simply  be  added  to  the  total  uncertainty  as 
outlined  in  Stem  et  al.  [1999].  When  it  occurs  on  the  coarser  grids  the  procedure  is  less  clear  and 
a  value  for  the  uncertainty  of  the  calculation  is  less  easily  obtainable.  When  performing  a  V&V 
analysis  on  a  large  field  of  data,  providing  iterative  convergence  for  all  values  can  be  unrealistic 
because  of  the  huge  number  of  data  points  involved.  Consequently,  it  is  recommended  that  the 
iterative  histories  of  a  representative  sample  of  the  quantities  being  verified  be  examined  to 
assess  iterative  convergence.  In  the  absence  of  a  clear  method  for  computing  an  iterative 
uncertainty  which  is  known  to  be  non-negligible,  it  is  recommended  that  an  effort  be  made  to 
provide  statements  regarding  where  the  iterative  convergence  uncertainty  is  significant  and, 
hence,  where  the  simulation  numerical  uncertainty  estimates  cannot  be  regarded  as  accurate. 

The  remainder  of  this  section  will.be  used  to  document  the  procedure  recommended  for 
estimating  the  grid  uncertainty  (Ug).  The  grid  uncertainty  is  an  estimate  of  the  confidence  level 
one  has  that  the  solution  has  achieved  grid  independence  based  on  comparisons  with  solutions 
from  coarser  grids.  Ideally  at  least  three  grids  would  be  used:  a  fine,  medium,  and  coarse  grid. 
The  numerical  results  to  be  validated  are  computed  for  all  grids,  including  any  integrated 
quantities,  and  any  field  data  that  is  to  be  directly  validated  must  be  interpolated  to  the  locations 
of  the  experimental  data.  Prior  to  estimating  the  grid  uncertainty  the  numerical  results  from  the 
related  grids  must  be  used  to  compute  the  convergence  ratio  from  which  the  convergence 
condition  is  determined.  The  convergence  condition  must  indicate  that  the  solutions  are 
converging  in  order  for  a  grid  uncertainty  estimate  to  be  made. 

Determination  of  the  Convergence  Condition 

For  validating  single  point  quantities,  or  integrated  quantities  that  are  associated  with  the 
entire  data  set  (i.e.  the  drag  of  a  body),  the  following  definition  for  a  quantity  known  as  the 
convergence  ratio  (R)  is  used. 

R  =  e2\/e32 
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The  quantities  e2i  and  e32  are  simply  the  differences  in  the  medium  and  fine  grid  predictions  and 
the  coarse  and  medium  grid  predictions  respectively  (i.e.  e2i=S2-Si  and  e32=S3-S2,  where  Si,  S2 
and  S3  are  the  predictions  from  the  fine,  medium  and  coarse  grids  respectively).  For  validating  a 
set  of  field  data  (i.e.  a  boundary  layer  profile,  surface  wave  height,  surface  pressure  map,  etc.)  it 
is  recommended  that  the  L2  norm  of  e2i  and  e32  be  computed  over  the  range  of  data  being 
considered  to  obtain  a  “global  convergence  ratio”  as  defined  below. 


where. 


W-hit/h*!* 


hA 


ife,)2 
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This  “global  convergence  ratio”  is  recommended  in  order  to  avoid  situations  where  a  few  ill 
conditioned  points,  within  the  profile  or  field  data  set,  are  prevented  from  undergoing  the 
verification  and  validation  process. 

Based  on  the  computed  value  of  R  or  (R)  the  convergence  condition  is  determined  as 
outlined  below. 


•  Integrated  or  single  point  quantities: 

R<0:  An  oscillating  condition  is  implied  by  these  3  particular  solutions 
R>1:  A  divergent  condition  is  implied  by  these  3  particular  solutions. 
0<R<1:  A  convergent  condition  is  implied  by  these  3  particular  solutions. 

•  Field  and  profile  data: 

(R)>1:  On  average  a  non-convergent  condition  is  implied  by  these  3  particular 
solutions. 

0<(R)<1:  A  convergent  condition  is  implied  by  these  3  particular  solutions. 

If  the  convergence  condition  is  determined  to  be  divergent/non-con vergent  (R>1  or  (R)>1)  no 
grid  uncertainty  estimate  can  be  made  using  the  procedure  of  Stem  et  al.  [1999].  Also,  if  the 
convergent  condition  is  determined  to  be  oscillating  (R<0)  no  grid  uncertainty  can  be  estimated 
without  performing  more  numerical  simulations.  Refer  to  Stem  et  al.  [1999]  for  further 
discussion  on  this  topic.  The  next  section  describes  the  recommended  methods  for  estimating 
the  grid  uncertainty  for  data  with  convergent  convergence  conditions  (0<R<1  or  0<(R)<1). 
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Estimation  of  the  Grid  Uncertainty 

In  this  report,  estimation  of  the  grid  uncertainty  is  performed  following  one  of  two 
methods.  The  first  estimate  is  based  on  the  methodology  of  Stem  et  al.  [1999]  and  at  its  core 
uses  Richardson  extrapolation.  The  following  equations  define  this  grid  uncertainty  estimate. 

UG  =  \CdRE\  +  \(l-C)dRE\ 

where, 

C  =  (rp  -1  )/(rPth  -l) 

dRE~e  2l/(rP_1) 

p  =  ln(l/R)/ln(r) 

r  -  grid  refinement  ratio 

pth  =  theoretical  order  of  accuracy 

e2i=S2-Si 

R  =  convergence  ratio 

In  these  equations,  C  is  referred  to  as  the  correction  factor,  p  as  the  order  of  accuracy  estimate, 
and  dRE  as  the  Richardson  extrapolation  error.  The  convergence  ratio  and  the  grid  refinement 
ratio  are  used  to  compute  the  order  of  accuracy  estimate.  Strictly  speaking,  the  grid  refinement 
ratio  is  defined  as  the  ratio  of  grid  spacing  between  two  successive  grids.  For  practical  problems 
the  grid  refinement  ratio  is  somewhat  difficult  to  define  due  to  the  non-uniformity  of  the  grids 
used  in  solving  such  problems.  Studies  found  in  the  literature  indicate  that  the  use  of  global 
parameters,  such  as  the  overall  number  of  grid  points,  to  compute  the  grid  refinement  ratio  can 
be  used  for  geometrically  similar  grids  that  make  appropriate  consideration  for  the  physics  of  the 
problem  [Celik,  1997].  The  equations  given  above  are  valid  only  when  the  grid  refinement  ratio 
is  constant  for  all  successive  grids.  The  order  of  accuracy  estimate  is  then  used  to  compute  the 
Richardson  extrapolation  error.  The  correction  factor  is  used  in  an  attempt  to  correct  for  poor 
order  of  accuracy  estimates,  which  can  occur  when  one  or  more  of  the  solutions  is  not  in  the 
asymptotic  range.  Again,  refer  to  Stem  et  al.  [1999]  for  a  detailed  discussion  of  these  equations 
and  their  origins.  The  next  section  illustrates  the  use  of  this  grid  uncertainty  estimation  method. 

Example:  Verification  and  Validation  of  a  Wave  Profile 

The  procedure  outlined  above  is  applied  here  to  a  predicted  wave  height  profile  obtained 
with  the  Reynolds  Averaged  Navier-Stokes  (RANS)  code  UNCLE  [Taylor,  1991]  on  three 
related  grids.  The  coarse  and  medium  grids  were  obtained  by  removing  every  other  point  from 
the  next  finer  grid,  giving  a  grid  refinement  ratio  of  2.0.  Figure  1  provides  a  comparison 
between  the  fine  grid  numerical  prediction  and  the  experimentally  measured  wave  profile.  The 
wave  height  profiles  predicted  by  the  three  different  grids  are  shown  in  Figure  2.  For  this 
example  the  L2  norms  of  e2i  and  e32  are  computed  over  the  range  of  data  points  that  make  up  the 
wave  profile  to  come  up  with  a  “global  convergence  ratio”. 
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W  =  hl||2/ih2||2  =0.695 

Based  on  this  computation  it  is  determined  that  a  convergent  condition  is  implied  by  these 
3  particular  solutions,  hence  the  grid  uncertainty  can  be  estimated  using  the  method  outlined  in 
the  previous  section.  The  UNCLE  code  uses  a  3rd  order  upwind  scheme  for  the  convective  terms 
and  a  2nd  order  central  difference  scheme  for  the  viscous  terms.  Realistically,  because  of  grid 
stretching  and  other  factors  one  would  normally  expect  the  order  of  accuracy  of  the  scheme  to  be 
between  1st  and  2nd  order.  Therefore,  the  theoretical  estimate  of  the  order  of  accuracy  for  this 
particular  example  is  chosen  to  be  2.  Applying  this  method  produces  the  following  results. 

r  =  2 


Pth= 2 
R  =  0.695 
p  =  0.525 
C  =  0.146 

From  these  numbers  the  grid  uncertainty  is  estimated  at  all  of  the  points  which  make  up  the 
interpolated  fine  grid  wave  profile  using  the  equations  (repeated  below)  given  in  the  procedure. 

UG  =\CdRE\  +  \{l-C)dRE\ 

dRE  =e2l/[rP  _1) 

Note  that  even  if  a  “global  convergence  ratio”  is  used,  the  quantity  e2i  in  the  equation  for 
the  Richardson  extrapolation  error  is  computed  individually  for  each  point.  Figure  3  indicates 
the  result  of  this  computation  by  plotting  the  fine  grid  solution  along  with  two  lines  that  represent 
the  estimated  grid  uncertainty  band  for  that  solution.  The  validation  procedure  is  carried  through 
using  the  estimated  grid  uncertainty  and  an  experimental  uncertainty  of  7.72xl0'4,  which  was 
provided  by  the  experimentalist.  For  this  example  the  iterative  convergence  uncertainty  (Ui)  has 
been  ignored.  A  comparison  between  the  validation  uncertainty  and  the  absolute  error  between 
the  computation  and  experiment  is  shown  in  Figure  4.  It  can  be  seen  that  for  the  most  part  the 
figure  indicates  a  validated  solution  “at  the  Uv  level.” 

This  method  for  estimating  the  grid  uncertainty  requires  three  related  grids  and  is  founded  on  the 
assumption  that  these  three  grids  produce  solutions  that  are  in  the  asymptotic  range.  This  is  a 
very  difficult  requirement  for  a  highly  complicated  geometry.  While  evaluating  the  RANS 
computations  obtained  with  UNCLE  it  became  apparent  that  the  coarsest  grid  produces  solutions 
that  are  definitely  outside  of  the  asymptotic  range,  at  least  for  the  wave  height  example 
documented  here.  This  can  be  inferred  from  the  order  of  accuracy  estimate,  p,  obtained  from  the 
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Fig.  3.  Fine  grid  numerical  prediction  and  the  grid  uncertainty  band  as  estimated  using  the 

procedure  of  Stem  et  al.  [1999]. 


Fig.  4.  Validation  plot  obtained  with  the  grid  uncertainty  estimated  using  the  procedure  of  Stem 

etal.  [1999].  * 


three  solutions.  As  already  discussed,  one  would  normally  expect  the  order  of  accuracy  of  the 
scheme  to  be  between  1st  and  2nd  order.  Although  it  is  possible  to  get  orders  of  accuracy 
somewhat  less  than  theoretical,  a  value  of  0.525  seems  unrealistically  low  particularly  for  the 
medium  and  fine  grids  used  here.  On  top  of  this,  we  know  that  the  equation  for  p 
(p=ln(l/R)/ln(r))  is  a  poor  estimate  for  the  order  of  accuracy  when  all  three  grids  are  not  in  the 
asymptotic  range.  The  correction  factor,  C,  introduced  by  Stem  et  al.  [1999]  is  an  attempt  to 
correct  for  values  of  p  that  are  well  outside  of  the  expected  theoretical  range.  This  correction 
factor  seems  to  have  the  correct  type  of  effect  for  high  values  of  p  by  making  more  conservative 
estimates  on  the  numerical  uncertainty.  It  can  be  seen  from  the  equation  for  the  estimated  grid 
uncertainty  that. 

If  p»pth  (C»l)  =>  UG~2ICdREl 

However,  for  values  of  p  that  are  less  than  the  theoretical  order  of  accuracy  the  nature  of  the 
equation  for  the  estimated  uncertainty  is  such  that  no  correction  at  all  is  made. 

If  p<pth  (C<1)  =>  Ug  =  ldREl 

This  is  the  correct  behavior  for  p  sufficiently  close  to  p,h.  However,  this  can  lead  to  overly 
conservative  estimates  for  the  uncertainty  on  the  numerical  solution  when  p  is  significantly 
below  the  theoretical  value.  This  appears  to  be  the  case  for  this  example,  as  seen  from  Figure  3. 
It  can  be  argued  that  this  method  for  estimating  the  grid  uncertainty  has  produced  an  unfairly 
conservative  estimate  owing  to  the  inclusion  of  solutions  that  are  not  in  the  asymptotic  range. 

Additionally,  using  the  L2  approach  for  the  estimate  of  R  can  give  the  impression  a 
solution  is  not  converging  when  it  really  is.  The  problem,  again,  is  because  the  R  computation  is 
a  measure  of  convergence  based  on  all  three  grids.  If  the  coarsest  grid  is  outside  the  asymptotic 
range  one  can  easily  obtain  a  value  of  R  that  implies  the  solution  is  not  converging,  preventing 
an  uncertainty  estimate  from  being  obtained  with  this  approach.  Consequently,  it  is 
recommended  to  use  the  approach  of  Stem  et  al.  [1999]  outlined  above  where  it  seems  realistic. 
However,  an  alternative  method  is  recommended  in  situations  where  the  set  of  solutions  do  not 
meet  all  the  criteria  required  for  obtaining  meaningful  estimates  of  the  grid  uncertainty  using  the 
first  method  or  when  no  estimate  can  be  made  at  all.  This  alternative  method  is  presented  in  the 
following  section. 

Alternative  Estimation  of  Grid  Uncertainty  Based  on  2  Grids 

A  recommended  alternative  estimate  of  the  grid  uncertainty  is  made  based  on  an 
extrapolation  using  only  the  medium  and  the  fine  grid  solutions.  Although  three  grids  are  needed 
to  demonstrate  convergence,  only  two  grids  are  needed  to  estimate  sensitivity.  Consequently  it  is 
felt  that  a  better  estimate  of  the  uncertainty  on  the  fine  grid  solution  for  this  particular  case  would 
be  to  assume  that  the  scheme  is  1st  order  accurate  and  compute  the  uncertainty  from  the  two 
finest  grids.  Again  this  estimate  is  still  on  the  conservative  side.  This  approach  should  be  used 
when  the  estimate  of  p  from  the  three  grids  is  less  than  1 .  It  can,  however,  also  be  used  when  the 
computed  p  is  significantly  greater  than  2.  The  basic  assumption  here  is  that  the  coarsest  grid  is 
an  unrealistic  prediction  of  the  flow  field  and  the  medium  and  fine  grids  seem  reasonable  and 
consistent  with  each  other.  It  is  also  assumed  here  that  the  fine  and  medium  grids  appear  to  be 
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converging  to  a  particular  solution.  The  equations  for  the  uncertainty  estimate  reduce  to  the 
following  equation. 

UG=‘2\l[rPesl  -l) 

where, 

Pest  =  1 

r  =  grid  refinement  ratio 

^21=S2-Sl 

This  produces  a  grid  uncertainty  that  is  equivalent  to  the  Grid  Convergence  Index  of  Roache 
[1998]  for  second  order  accurate  solutions  using  the  recommended  “factor  of  safety”  of  3. 

Applying  this  approach  to  the  predicted  wave  height,  of  the  current  example,  results  in  the 
estimated  grid  uncertainty  band  shown  in  Figure  5  and  corresponding  validation  comparison  of 
Figure  6.  The  uncertainty  band  on  the  solution  in  Figure  5  is  considerably  smaller  with  this 
alternative  method  than  that  shown  in  Figure  3  even  though  the  solutions  on  the  fine  and  medium 
grids  are  exactly  the  same  as  before.  This  shows  that  the  order  of  accuracy  estimate,  p,  is 
driving  the  uncertainty  bands.  For  the  most  part,  both  methods  indicate  a  validated  solution  “at 
the  Uv  level,”  however,  it  is  of  interest  to  note  that  by  using  the  less  conservative  estimate  for  the 
grid  uncertainty,  one  actually  obtains  a  more  conservative  estimate  of  Uv. 


Fig.  5.  Fine  grid  numerical  prediction  and  the  grid  uncertainty  band  as  estimated  using  the 

alternative  procedure. 
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Fig.  6.  Validation  plot  obtained  with  the  grid  uncertainty  estimated  using  the  alternative 

procedure. 

Conclusions 

This  report  has  presented  a  procedure  for  performing  a  verification  and  validation  analysis  for 
computational  fluid  dynamic  solutions.  The  procedure  requires  the  estimation  of  uncertainties 
from  many  sources.  In  general  it  is  advisable  to  eliminate,  or  render  negligible,  as  much  of  the 
uncertainty  as  possible.  In  particular  iterative  convergence  uncertainty  is  difficult  to  quantify 
and  requires  further  study.  In  contrast,  the  estimation  of  the  uncertainty  due  to  the  use  of  finite 
size  grids  has  a  relatively  strong  theoretical  background.  However,  issues  of  a  practical  nature 
arise  when  making  an  estimation  of  the  grid  uncertainty  for  problems  with  complex  geometry 
and  physics.  Problems  occur  when  one  or  more  of  the  solutions  are  not  in  the  asymptotic  range 
and  the  estimated  order  of  accuracy  is  unrealistic.  The  approach  of  Stem  et  al.  [1999]  attempts 
to  remedy  this  problem  through  the  use  of  a  correction  factor.  It  has  been  shown  in  this  report 
that  the  correction  factor  is  appropriate  in  situations  where  the  order  of  accuracy  estimate  is 
unrealistically  high,  but  can  lead  to  overly  conservative  estimates  of  uncertainty  when  the  order 
of  accuracy  is  estimated  to  be  significantly  below  the  theoretical  value.  An  alternative  method 
for  estimating  the  grid  uncertainty  has  been  recommended  for  these  situations,  which  requires 
solutions  on  only  two  grids  and  uses  a  conservative  theoretical  value  for  the  order  of  accuracy. 

Point-to-point  comparison,  as  is  done  while  validating  field  data  following  this  procedure, 
can  be  problematic  for  data  near  inflection  points  where  small  changes  in  phase  or  position  are 
not  readily  accounted  for.  Consequently,  one  should  not  loose  sight  of  the  traditional 
comparisons  of  experimental  data  and  computations  and  base  everything  on  the  V&V  results.  It 
should  be  recognized  that  using  numerical  predictions  at  this  level  and  using  verification  and 
validation  methods  of  this  type  is  really  in  its  infancy  and  will  most  likely  evolve  over  time. 
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